System and Method for Evaluating a Driving Behavior

ABSTRACT

A system and computer-implemented method detect and act upon operator responsiveness to vehicle alerts. The system and method include determining that an Advanced Driver Assistance system (ADAS) of a vehicle generated an alert for a driving activity. The system and method may include logging a first set of measurements data associated with the alert of the ADAS in a database saved in a memory. The system and method may compare a portion of the first set of measurements data with a portion of the second set of measurements data, determine a driving metric for an operator of the vehicle based upon the comparing, and set at least a portion of an operator profile associated with the operator with the driving metric. As a result, risk averse driver, and/or proper responsiveness to vehicle alerts may be rewarded with insurance-cost savings, such as increased discounts.

CROSS REFERENCE TO RELATED APPLICATIONS

This present disclosure claims the benefit of U.S. Provisional Patent Application No. 62/563,722, entitled “System and Method for Evaluating Driving Behavior” and filed on Sep. 27, 2017; U.S. Provisional Patent Application No. 62/563,729, entitled “Evaluating Operator Reliance on Vehicle Alerts” and filed on Sep. 27, 2017; U.S. Provisional Application No. 62/563,808, entitled “Automatically Tracking Driving Activity” and filed on Sep. 27, 2017; and U.S. Provisional Application No. 62/563,818, entitled “Automated Selection of a Vehicle” and filed on Sep. 27, 2017, all of which are incorporated herein in by reference in their entirety.

TECHNICAL FIELD

The present disclosure relates generally to evaluating driving behavior for a particular driving activity. More particularly, the present disclosure relates to evaluating an operator's response to alerts provided by an Advanced Driver Assistance System (ADAS) installed in the driven vehicle.

BACKGROUND

An Advanced Driver Assistance System (ADAS) installed in a vehicle may aid the operator of the vehicle by providing alerts in response to an operator's actions. In general, an ADAS may monitor various traffic conditions and/or the external environment surrounding the vehicle, may take measurements of objects using radar or camera-based sensors, to assist the operator.

An example of an ADAS is a blind spot monitoring system. A blind spot monitoring system may provide alerts to an operator if a vehicle-based sensor device detects other vehicles located to the operator's side and/or rear, which may aid the operator when changing lanes. Another example of an ADAS is a lane departure warning system. A lane departure warning system may provide alerts to an operator if a vehicle-based sensor device detects that the vehicle is beginning to move out of its lane, which may aid the operator to stay in his or her lane. Other examples of an ADAS may include a forward collision warning system. However, driver reliance on ADAS systems may vary by individual, which may cause one or more drawbacks.

BRIEF SUMMARY

The present embodiments disclose systems and methods that may generally relate to evaluating driving behavior for a particular driving activity, and particularly, inter alia, to evaluating an operator's response to alerts provided by an Advanced Driver Assistance System (ADAS) installed in the driven vehicle.

In one aspect, a computer-implemented method for detecting and acting upon operator responsiveness to vehicle alerts may be provided. The method may include: (1) determining, by the processor, that an Advanced Driver Assistance System (ADAS) of a vehicle generated an alert for the driving activity; (2) logging, by the processor, a first set of measurements data associated with the alert of the ADAS in a database saved in a memory, or in a memory unit; (3) receiving, by the processor, a second set of measurements data in response to the alert for the driving activity; (4) comparing, by the processor, a portion of the first set of measurements data with a portion of the second set of measurements data; (5) determining a driving metric for an operator of the vehicle based upon the comparing; and/or (6) setting, by the processor, at least a portion of an operator profile associated with the operator with the driving metric. As a result, proper responsiveness or unresponsiveness to vehicle alerts by risk averse drivers may be rewarded, such as with lower insurance premiums or increased insurance discounts. The method may include additional, less, or alternate actions, including those discussed elsewhere herein.

In another aspect, a computer system for detecting and acting upon operator responsiveness to vehicle alerts may be provided. The system may include one or more processors, transceivers, and memory units storing instructions. When executed by the one or more processors, the instructions may cause the computer system to: (1) determine that an Advanced Driver Assistance System (ADAS) of a vehicle generated an alert for the driving activity; (2) log a first set of measurements data associated with the alert of the ADAS in a database saved in one or more memory units; (3) receive a second set of measurements data in response to the alert for the driving activity; (4) compare a portion of the first set of measurements data with a portion of the second set of measurements data; (5) determine a driving metric for an operator of the vehicle based upon the comparing; and/or (6) set at least a portion of an operator profile associated with the operator with the driving metric. The computer system may include additional, less, or alternate functionality, including that discussed elsewhere herein.

In another aspect, a non-transitory, computer-readable medium (or media) stores instructions that, when executed by one or more processors, cause the one or more processors to: (1) determine that an Advanced Driver Assistance System (ADAS) of a vehicle generated an alert for the driving activity; (2) log a first set of measurements data associated with the alert of the ADAS in a database saved in a memory; (3) receive a second set of measurements data in response to the alert for the driving activity; (4) compare a portion of the first set of measurements data with a portion of the second set of measurements data; (5) determine a driving metric for an operator of the vehicle based upon the comparing; and/or (6) set at least a portion of an operator profile associated with the operator with the driving metric. The instructions may direct additional, less, or alternate functionality, including that discussed elsewhere herein.

Advantages will become more apparent to those skilled in the art from the following description of the preferred embodiments which have been shown and described by way of illustration. As will be realized, the present embodiments may be capable of other and different embodiments, and their details are capable of modification in various respects. Accordingly, the drawings and description are to be regarded as illustrative in nature and not as restrictive.

BRIEF DESCRIPTION OF DRAWINGS

The Figures described below depict various aspects of the systems and methods disclosed therein. It should be understood that each Figure depicts an embodiment of a particular aspect of the disclosed systems and methods, and that each of the Figures is intended to accord with a possible embodiment thereof. Further, wherever possible, the following description refers to the reference numerals included in the following Figures, in which features depicted in multiple Figures are designated with consistent reference numerals.

There are shown in the drawings arrangements which are presently discussed, it being understood, however, that the present embodiments are not limited to the precise arrangements and are instrumentalities shown, wherein:

FIG. 1 is a block diagram of an exemplary computer system for identifying driving behavior, and generating, modifying, and/or using driver profiles;

FIG. 2 depicts exemplary data categories that may be utilized by the computer system of FIG. 1 to identify driving behavior, and/or generate or modify an operator profile;

FIG. 3 depicts exemplary profile information categories that may be determined by the computer system of FIG. 1 when identifying driving behavior patterns and/or generating or modifying an operator profile;

FIG. 4 depicts an exemplary scenario in which an operator followed a valid ADAS alert;

FIG. 5 depicts an exemplary scenario in which a false positive ADAS alert was issued;

FIG. 6 is a flow diagram of an exemplary computer-implemented method for detecting and acting upon operator responsiveness to alerts provided by ADAS; and

FIG. 7 is a block diagram of an exemplary computer system on which one or more of the embodiments described herein may be implemented.

The Figures depict aspects of the present invention for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternate aspects of the structures and methods illustrated herein may be employed without departing from the principles of the invention described herein.

DETAILED DESCRIPTION

The embodiments described herein relate to, inter alia, systems and techniques for identifying driving behavior, and/or generating, modifying, and/or using profiles for drivers/operators of vehicles. The operator profile may be generated and/or modified using vehicle telematics data indicative of how the operator/operator drives the vehicle (e.g., acceleration data, braking data, cornering data, etc.), data indicative of when and/or where the operator/operator drives the vehicle (e.g., GPS data), data indicative of the circumstances in which the operator/operator drives the vehicle (e.g., camera or other sensor data indicating the presence of passengers in the vehicle, the distance between the operator's vehicle and other vehicles, weather, time of day, geographic information, etc.), and/or other data (e.g., demographic information, dealership information regarding recalls or maintenance, etc.).

As the term is used herein, “vehicle telematics data” may include any suitable type or types of data provided by the vehicle (e.g., one or more sensors and/or subsystems of the vehicle), by a mobile electronic device carried or located within the vehicle (e.g., a smartphone or wearable electronic device of the operator), and/or by any other electronic device or component carried on or within the vehicle. Depending upon the context and the embodiment, for example, vehicle telematics data may include acceleration data generated by an electronic control system of the vehicle and/or by an accelerometer of the operator's mobile electronic device, GPS data provided by a GPS unit of the vehicle and/or a GPS unit of the mobile electronic device, speed and acceleration data, heading and direction data, route information, image or video data generated by a camera of the vehicle and/or a camera of the mobile electronic device, and so on.

In various different embodiments, the operator profiles may be used in different situations or scenarios. For example, the profiles may be used to adjust a price to risk model or insurance ratings (e.g., during initial underwriting, or when renewing a policy, etc.), to rate or showcase a driving instructor or student, to determine whether to provide a discount, and/or for other purposes. In some embodiments, the profile may include a rating that is indicative of the operator's personal responsibility or trustworthiness, and may be used in situations where such qualities are of particular importance. For example, driving metrics in driver profiles may be used to adjust driver credit ratings (e.g., when applying for a loan or credit line), to determine whether an “IOU” may be accepted from an individual, to determine whether candidates will be offered particular jobs, and so on.

In one embodiment, an ideal responsible driver may be characterized as an operator who does not over-rely on ADAS when driving, but rather, uses ADAS as an aid in appropriate situations. An operator that is over-reliant on ADAS may not be establishing or maintaining his or her general driving behavior. If a particular ADAS overly-relied upon is malfunctioning or inoperable, or if a rental vehicle does not feature the particular ADAS overly-relied upon, the operator may not be prepared to safely drive the vehicle.

Additionally or alternatively, an ideal responsible driver may be characterized as an operator who minimally uses ADAS when driving, as opposed to never using ADAS at all, in appropriate situations. For example, if the operator's natural driving behavior of staying within a lane on a highway needs further training or practice, using a lane departure warning system may be safer than not using one at all, if there are no indications that the lane departure warning system in the vehicle is malfunctioning or inoperable, or providing erroneous alerts. The present embodiments may measure an operator's response to alerts from an ADAS that may be highly probative of an operator's characteristics or qualities as it relates to risk averse driving behavior, and may incorporate such measurements into evaluating driving behavior.

Exemplary Aspects

The present embodiments may relate to customer selection and use of ADAS systems, and/or pricing insurance to corresponding operator risk models or profiles. A number of hypotheses resolve around ADAS. For instance, good drivers may become bad drivers as they become dependent on ADAS instead of their natural good driving skills. For example, a driver may become reliant on a blind spot indicator instead of looking over their shoulder.

On the other hand, bad drivers may become artificially better drivers as they rely on ADAS systems to tell them when they are performing poorly and correct driver actions automatically. For example, emergency breaking may occur frequently, although no collision occurs.

Because of multiple false positives, a potentially unsafe environment may be created where the driver acknowledges the warning exists, but ignores taking action or deems not critical/accurate based upon previous experiences. For example, ADAS may ask the driver to place their hands on wheel due to an impending driver takeover. However, because of multiple false positives where no action was actually necessary, the driver knowingly ignores and the vehicles crashes.

Also, ADAS warning alerts may be ignored when drivers interpret the warning as not important or critical regardless of ADAS accuracy. For example, ADAS may warn the driver, and indicated that human takeover from autopilot is warranted. However, past driver conditioning causes the driver to deem the warning as not critical, increasing the likelihood of a vehicle collision.

Therefore, as ADAS adoption increases, ADAS dependence and/or reliance by drivers may result in difficulties in accurately determine potential or actual risk with safety features, vehicles, and/or operators.

The present embodiments may relate to collecting various data points to create a price to risk model. For instance, the data points may relate to: (i) previous driving/claim history; (ii) build sheet data; (iii) # of ADAS features equipped on vehicle; (iv) ADAS on/off (manual vs auto); (v) history of ADAS system turned on/off (such as manually turning off Lane Departure Warning); (vi) collection of data showing how driver responds to or ignores ADAS warnings; (vii) # of times ADAS Triggers Per Vehicle Ignition (weighted Safety Impact and ROI on Safety); (viii) emergency breaking; (ix) lane keep; (x) lane departure warning; (xi) blind spot warning; (xii) front/rear cross traffic alert; (xiii) adaptive cruise control; (xiv) park assist; (xv) automatic high beam; (xvi) reaction baseline to ADAS engagement (how driver reacts to ADAS when enabled and activated); and/or (xvii) VR simulation to pre-rate.

Additionally or alternatively, the data points may relate to: knowledge and training on ADAS (or experience); speed; GPS location; frequency of vehicle rental annually; frequency of vehicle sharing annually; quality of vehicle brand and ADAS system; ambient traffic conditions (density, speed, time of day); rating should decrease over time as technology improves to newest model year; miles driven annually or times of day that commute typically happens; and/or exhibiting traits over time of good or preferred driving behaviors, especially in risky or heavy traffic environments.

Exemplary Computer System for Generating, Modifying, and/or Using Driver Profiles

FIG. 1 is a block diagram of an exemplary system 10 for identifying driving behavior, and generating, modifying, and/or using driver profiles or operator profiles. The system 10 may include a vehicle 12 having an on-board system 14, as well as a computer system 16, a third party server 18, and a network 20. While FIG. 1 depicts vehicle 12 as an automobile, vehicle 12 may instead be a truck, motorcycle, or any other type of land-based vehicle capable of carrying at least one human passenger (including the operator).

Network 20 may be a single wireless network, or may include multiple cooperating and/or independent networks of one or more types (e.g., a cellular telephone network, a wireless local area network (WLAN), the Internet, etc.). On-board system 14 and third party server 18 may both be in communication with the computer system 16 via network 20. While FIG. 1 shows only a single vehicle 12 and a single third party server 18, it is understood that computer system 16 may communicate (e.g., via network 20) with any number of different vehicles (e.g., one or more other vehicles similar to vehicle 12, with on-board systems similar to on-board system 14) and/or any number of different third party servers.

Third party server 18 may be a server of an entity that is not affiliated with either the operator of vehicle 12 or the entity owning, maintaining, and/or using computer system 16, and may be remote from computer system 16 and/or vehicle 12. For example, in various different embodiments discussed further below, third party server 18 may be a server associated with a provider of a mapping service, a provider of a weather information service, an auto repair shop, an auto maker, an auto dealership, an auto parts supplier, an entity that determines credit scores, and so on. As used herein, the term “server” may refer to a single server, or multiple servers communicating with each other.

On-board system 14 may include a first external sensor 30 and a second external sensor 32, each being configured to sense an environment external to vehicle 12 (i.e., to sense physical characteristics of the environment external to vehicle 12), such as a still image or video camera device, a lidar (laser remote sensing, or light detection and ranging) device, a radar device, or a sonar device, for example. Each of the external sensors 30, 32 may be located on or inside vehicle 12. For example, one or both of the external sensors 30, 32 may be permanently affixed to vehicle 12 (e.g., on the exterior or interior of the frame, on the dashboard, on the inner or outer surface of a windshield, etc.), or may be temporarily affixed to, or simply placed on or in, some portion of the vehicle 12 (e.g., placed on top of the dashboard, or in a device holder affixed to the windshield, etc.).

External sensor 30 and/or external sensor 32 may be included in a general purpose computing device (e.g., as a software application and associated hardware of a smartphone or other portable computer device), or may be a dedicated sensor device. In the exemplary system 10 shown in FIG. 1, external sensor 30 is located on or inside vehicle 12 such that it senses the environment in a forward-facing range 34, while external sensor 32 is located on or inside vehicle 12 such that it senses the environment in a rear-facing range 36. In some embodiments, the external sensor 30 and external sensor 32 may collectively provide a 360 degree sensing range. In other embodiments, however, the external sensor 30 and external sensor 32 may be redundant sensors (of the same type, or of different type) that each provide a 360 degree sensing range. In still other embodiments, either the external sensor 30 or external sensor 32 may be omitted, or the on-board system 14 may include more than two external sensors.

Each of external sensors 30, 32 may generate data, or analog information, that is indicative of the sensed external environment. In one embodiment where external sensor 30 is a digital video camera device, for example, external sensor 30 may generate data corresponding to frames of captured digital video. As another example, in one embodiment where external sensor 30 is a digital lidar device, external sensor 30 may generate data corresponding to frames of captured digital lidar information.

On-board system 14 may also include one or more internal sensors 38. In some embodiments, internal sensor(s) 38 may include one or more sensors designed to detect the presence of passengers. For example, internal sensor(s) 38 may include inward-facing digital cameras arranged to capture at least a portion of an interior (cabin) of vehicle 12, and/or one or more seat or weight sensors configured to detect the presence of the operator and/or passengers in the respective seat(s). As another example, internals sensor(s) 38 may instead (or also) include seatbelt sensors that are configured to detect when each seatbelt in vehicle 12 is engaged or not engaged. In certain embodiments where internal sensor(s) 38 include an inward-facing camera, the camera may be permanently affixed to vehicle 12 (e.g., on the interior of the frame, on the dashboard, on the inner surface of a windshield, etc.), or may be temporarily affixed to, or simply placed on or in, some portion of vehicle 12 (e.g., placed on top of the dashboard, or in a device holder affixed to the windshield, etc.). Moreover, a camera of internal sensor(s) 38 may be included in a general purpose computing device (e.g., as a software application and associated hardware of a smartphone or other portable computer device), or may be a dedicated sensor device.

On-board system 14 may also include an Advanced Driver Assistance System (ADAS) 22 that utilizes, but is not limited to utilizing, a braking subsystem 40, a speed subsystem 42, a steering subsystem 44, a diagnostics subsystem 46, and/or one or more different subsystems not shown in FIG. 1. By utilizing any one or more of the aforementioned subsystems and/or sensors (e.g., sensors 30, 32, 38), ADAS 22 may include a forward collision warning feature, a blind spot indication feature, a cruise control feature, a lane departure warning feature, an automatic high beam feature, and other advanced driver assistance features for vehicle 12.

For example, ADAS 22 including a forward collision warning feature may employ speed subsystem 42, a radar device, a lidar device, and/or a camera device (e.g., external sensor 30) to detect an imminent crash, and/or a GPS subsystem (e.g., GPS 48) to detect fixed dangers associated with a particular registered location, such as an approaching stop sign. In some embodiments, the GPS subsystem 48 may generate data indicative of a current location of vehicle 12, and in other embodiments, the subsystem 48 may use other positioning techniques instead of GPS, such as cell tower triangulation, for example.

Once the detection is done, ADAS 22 may either provide an alert to vehicle 12 (e.g., via diagnostics subsystem 46) when there is an imminent collision or take action autonomously without any driver input, such as by braking, slowing speed, and/or steering (e.g., via braking subsystem 40, speed subsystem 42, and/or steering subsystem 44, respectively). ADAS 22 may also generate contextual data that describes characteristics of driving behavior (e.g., speeding, accelerating, braking, lane shifting, weaving patterns, cornering, etc.) that led to either activation of the alert or the autonomous action, via braking subsystem 40, speed subsystem 42, steering subsystem 44, diagnostics subsystem 46, and/or one or more different subsystems not shown in FIG. 1. Similarly, ADAS 22 including a blind spot indication feature may employ external sensor 32 to sense the environment in a rear-facing range 36.

ADAS 22 may be a combination of hardware and software components that provides data that may use one or more of the aforementioned subsystems and/or sensors to provide driver assistance for various driving activities. Such subsystems may be hardware, firmware and/or software subsystems that monitor and/or control various operational parameters of vehicle 12. As shown in FIG. 1, the diagnostics subsystem 46 may provide an alert to vehicle 12 when ADAS 22 has detected an event, such as another vehicle near the front, rear, or side of vehicle 12, lane departure, cruise control, or any other event ADAS 22 is configured to detect. The diagnostics subsystem 46 may also generate other information pertaining to the operation of vehicle 12, such as alert information to indicate that one or more components of vehicle 12 is/are in need of replacement, an upgrade, and/or servicing. For example, diagnostics subsystem 46 may generate a service alert when tire pressure is low (e.g., based upon a signal from a tire pressure sensor not shown in FIG. 1), when the engine is overheating (e.g., based upon a temperature sensor in the engine compartment, also not shown in FIG. 1), when an oil change is recommended, and so on. Subsequent to, prior to, or independent of ADAS 22 providing an alert to the vehicle 12 when ADAS 22 has detected an event, the braking subsystem 40 may generate data indicative of how the brakes of vehicle 12 are applied (e.g., an absolute or relative measure of applied braking force, or a binary indicator of whether the brakes are being applied at all, etc.), the speed subsystem 42 may generate data indicative of how fast the vehicle 12 is being driven (e.g., corresponding to a speedometer reading, an accelerometer measurement, and/or an operator input such as depression of a gas pedal, etc.), and the steering subsystem 44 may generate data indicative of how the vehicle 12 is being steered (e.g., based upon the operator's manipulation of a steering wheel, or based upon automated steering control data, etc.).

The aforementioned braking subsystem 40, speed subsystem 42, steering subsystem 44, diagnostics subsystem 46, and/or one or more different subsystems not shown in FIG. 1 may also generate data indicating whether ADAS 22 has taken control autonomously (without any driver input) over the subsystems 40, 42, 44 for vehicle 12. For example, the speed subsystem 42 may generate data indicating whether a cruise control feature of ADAS 22 is currently activated, and/or the braking subsystem 40 or steering subsystem 44 may generate data indicating whether assisted braking and/or assisted steering features of ADAS 22 are currently activated. As other examples, a unit of on-board system 14 (e.g., diagnostics subsystem 46, or another unit not shown in FIG. 1) may generate data indicating whether vehicle 12 is in an automated transmission mode or a manual transmission mode, or whether the driving of vehicle 12 is currently subject to complete automated/machine control rather than manual (human) control.

In some embodiments, the on-board system 14 may not include one or more of the subsystems 40, 42, 44, 46, 48, one or both of external sensors 30 and 32, and/or internal sensor(s) 38, and/or the on-board system 14 may include additional devices or subsystems not shown in FIG. 1. Moreover, one or more subsystems in on-board system 14 may be included in a general purpose computing device, such as a smartphone. For example, the GPS subsystem 48 may include a software application running on a smartphone that includes the appropriate hardware (e.g., an antenna and receiver).

On-board system 14 may also include a data collection unit 50 configured to receive data and/or analog signals from external sensors 30, 32, internal sensor(s) 38, some or all of subsystems 40, 42, 44, 46, 48, and/or ADAS 22. The data collection unit 50 may collect the data and/or analog signals substantially in real time, and in any of various different ways, according to different embodiments. In some embodiments, for example, the data collection unit 50 may periodically sample data and/or analog signals from the various external sensors 30, 32, internal sensor(s) 38, subsystems 40, 42, 44, 46, 48, and/or ADAS 22, or be notified by the respective sensors or subsystems when new data is available.

In some embodiments, the data collection unit 50 may receive data from one or more of the external sensors 30, 32, internal sensor(s) 38, one or more of subsystems 40, 42, 44, 46, 48 and/or ADAS 22 via a wireless link, such as a Bluetooth link. Alternatively, one or more of subsystems 40, 42, 44, 46, 48, internal sensor(s) 38, external sensors 30, 32 and/or ADAS 22 may provide data to data collection unit 50 via messages placed on a controller area network (CAN) bus (not shown in FIG. 1) or other suitable bus type, and/or via an on-board diagnostics (OBD) system (also not shown in FIG. 1). For example, the data collection unit 50 may collect information from one or more of subsystems 40, 42, 44, 46 via one or more OBD ports. In some embodiments, the data collection unit 50 may collect data using a mix of interface and/or bus types (e.g., a Bluetooth interface to receive data from sensors 30, 32 and internal sensor(s) 38, an OBD port to receive data from ADAS 22 and/or the diagnostics subsystem 46, and a CAN bus to receive data from subsystems 40, 42, 44).

In some embodiments where one or more of external sensors 30, 32, internal sensor(s) 38, one or more of subsystems 40, 42, 44, 46, 48, and/or ADAS 22 generate analog signals, either the respective sensors/subsystems/ADAS or the data collection unit 50 may convert the analog information to a digital format. Moreover, the data collection unit 50 may convert data received from one or more of external sensors 30, 32, internal sensor(s) 38, one or more of subsystems 40, 42, 44, 46, 48, and/or ADAS 22 to different digital formats or protocols. After collecting (and possibly converting) the data from the various sensors/subsystems/ADAS, the data collection unit 50 may store the data in a memory 52. The memory 52 may be any suitable type of data storage, such as a random access memory (RAM), a flash memory, or a hard drive memory, for example.

On-board system 14 may also include a data processing unit 54 that is coupled to the data collection unit 50. The data processing unit 54 may include one or more processors, or represent software instructions that are executed by one or more processors of on-board system 14, and may be configured to process the data collected by data collection unit 50 and stored in memory 52 for various purposes. In one embodiment, for example, data processing unit 54 simply packages data collected by data collection unit 50 into a format suitable for transmission to computing system 16. Alternatively, or in addition, data processing unit 54 may analyze the collected data to generate various types of information that may be used to update an operator profile, as discussed further below in connection with computing system 16. Data processing unit 54 may include, or be associated with, a memory 56 for storing outputs of the data analysis and/or other processing. Memory 56 may be any suitable type of data storage, such as a RAM, a flash memory, or a hard drive memory, for example. Memory 52 and memory 56 may be separate memories, or parts of a single memory, according to different embodiments.

Data processing unit 54 may be coupled to an interface 60, which may transmit the data received from data processing unit 54 to computer system 16 via network 20. Interface 60 may include a transmitter and one or more antennas, for example. In an alternative embodiment, interface 60 may instead be an interface to a portable memory device, such as a portable hard drive or flash memory device. In this embodiment, the portable memory device may be used to download data from memory 56 of data processing unit 54 or memory 52 of data collection unit 50, and may be manually carried to computer system 16 without utilizing network 20. In another alternative embodiment, a Bluetooth or other short-range link may be used to download data from memory 56 or memory 52 to a portable computer device (e.g., a laptop or smartphone), which may in turn be used to transmit the data to computer system 16 via network 20. In some embodiments, interface 60 may represent multiple types of different interfaces used for different types of data (e.g., a W LAN transceiver for data from external sensors 30, 32, a smartphone cellular transceiver for data from internal sensor(s) 38, and a flash memory device port for data from subsystems 40, 42, 44, 46, 48 and ADAS 22).

In some embodiments, the data generated by data processing unit 54 and stored in memory 56 may be automatically sent to interface 60 for transmission to computer system 16. For example, the data may be sent to interface 60 at regular time intervals (e.g., once per day, once per hour, etc.). In other embodiments, the data may be sent to computer system 16 in response to a query from computer system 16 that is received via network 20, or in any other suitable manner. Once the data is provided to computer system 16, the data may be subject to further processing to evaluate driving behavior for a particular driving activity (e.g., to determine a driving metric, to generate or modify a profile for the operator of vehicle 12 with the determined driving metric), as discussed further below.

Computer system 16 may be an electronic processing system (e.g., a server) capable of performing various functions, and may include an interface 62 configured to receive data from on-board system 14 of vehicle 12, and data from third party server 18, via network 20. Interface 62 may be similar to interface 60 of on-board system 14, for example. In certain embodiments where a portable memory device (rather than network 20) is used to transfer at least some of the data from on-board system 14 to computer system 16, interface 62 may include an interface to a portable memory device, such as a portable hard drive or flash memory device, for example.

Computer system 16 may also include a data collection unit 70 coupled to interface 62. Data collection unit 70 may be configured to receive/collect the data received by interface 62, and to store the collected data in a memory 72. Memory 72 may be any suitable type of data storage, such as a RAM, a flash memory, or a hard drive memory, for example. Data collection unit 70 may be coupled to a data analysis unit 74. Data analysis unit 74 may include one or more processors, or software instructions that are executed by one or more processors of computing system 16, and may be configured to process the data collected by data collection unit 70 and stored in memory 72 for various purposes according to different embodiments, as discussed further below.

Generally, data analysis unit 74 may analyze data from vehicle 12 (e.g., the data received from on-board system 14 via interface 60) and a number of other vehicles stored in historical driving data database 78 to determine whether alerts from ADAS 22 are false positive alerts or valid alerts, evaluate driving behavior (e.g., how an operator of vehicle 12 responded to false positive alerts, or erroneous or invalid alerts), determine a driving metric based upon the driving behavior, and generate and/or modify/update driver profiles stored in an operator profiles database 76 with the driving metric. Driver profiles database 76 and historical driving data database 78 may be stored in memory 72 or may be stored external to computer system 16 (e.g., memory 52).

In the exemplary system 10 of FIG. 1, data analysis unit 74 may include a driving behavior identification unit 80 and a profile generation/update unit 82. In other embodiments, data analysis unit 74 does not include one or more of units 80, 82 and/or includes additional units not shown in FIG. 1. For example, one or more of units 80, 82 may instead be implemented by data processing unit 54 of on-board system 14 in vehicle 12, or may be entirely absent from system 10. In one embodiment, each of units 80, 82 may include a set of instructions stored on a tangible, non-transitory computer-readable medium and capable of being executed by one or more processors of computer system 10 to perform the functions described below. In another embodiment, each of units 80, 82 may include a set of one or more processors configured to execute instructions stored on a tangible, non-transitory computer-readable medium to perform the functions described below.

Driving behavior identification unit 80 may be generally configured to analyze or process data received from vehicle 12 (e.g., from interface 60 of on-board system 14, as discussed above) to detect and/or identify various types of driving behaviors, as listed in driving behavior information 152 in FIG. 3. For example, driving behavior identification unit 80 may analyze data generated by any individual sensor, such as external sensor 30, to determine an average (and/or a minimum, etc.) tailgating distance between vehicle 12 and other vehicles in front of vehicle 12, and/or to determine proper/improper lane usage, etc.

As another example, driving behavior identification unit 80 may determine a first set of acceleration, braking, and/or weaving patterns of the operator of vehicle 12 and tag that set as being associated with the presence of one or more accompanied passengers, and determine a second set of acceleration, braking, and/or weaving patterns of the operator of vehicle 12 and tag that set as being associated with an absence of accompanied passengers. Driving behavior identification unit 80 may determine which sets correspond to the presence of one or more accompanied passengers using data generated by internal sensor(s) 38, for example.

Such sets of information may be probative of different driving behaviors associated with the operator of the vehicle 12 depending on whether other passengers are present in the vehicle 12, and may also be probative for the driving behavior identification unit 80 to determine how the operator responds to an alert provided by ADAS 22 depending on whether other passengers are in the vehicle 12. For example, upon analysis of the vehicle data from vehicle 12 via interface 60, if the driving behavior identification unit 80 determines that the first set of acceleration, braking, and/or weaving data of the operator of vehicle 12 shows a change in data when compared to vehicle data (e.g., data generated by subsystems 40, 42, 44, and/or 46 of ADAS 22) that caused a valid ADAS alert to be generated at the vehicle 12, driving behavior identification unit 80 may determine that the operator exhibited safe driving behavior by abiding by the valid ADAS alert. If the driving behavior identification unit 80 determines that the second set of acceleration, braking, and/or weaving data of the operator of vehicle 12 does not show a change in data when compared to vehicle data (e.g., data generated by subsystems 40, 42, 44, and/or 46 of ADAS 22) that caused a valid ADAS alert at the vehicle 12, driving behavior identification unit 80 may determine that a valid ADAS alert was not followed. Based upon a comparison of driving behaviors associated with the first and second sets, driving behavior identification unit 80 may also determine that the operator exhibits safer driving behavior when in the presence of one or more accompanied passengers.

As another example, driving behavior identification unit 80 may determine a first average tailgating distance of the operator of vehicle 12 and tag that distance as being associated with icy road conditions, determine a second average tailgating distance of the operator of vehicle 12 and tag that distance as being associated with wet road conditions, and/or determine a third average tailgating distance of the operator of vehicle 12 and tag that distance as being associated with dry road conditions. Driving behavior identification unit 80 may determine which distances correspond to the presence of icy, wet, or dry roads using data generated by external sensors 30 and/or 32, and/or data from a weather information service (e.g., in an embodiment where third party server 18 or another server not shown in FIG. 1 is associated with the weather information service), for example. Such tailgating distance information may be probative of different driving behaviors associated with the operator of the vehicle 12 depending on various road conditions, and may also be probative for the driving behavior identification unit 80 to determine how the operator responds to an alert provided by ADAS 22 in such road conditions.

Driving behavior identification unit 80 may be generally configured to analyze or process data received from stored historical data 78 to determine whether alerts from diagnostics subsystem 46 of ADAS 22 are false positive alerts or valid alerts. For example, if diagnostics subsystem 46 of ADAS 22 provides an alert at a location where historically, as indicated by the stored historical data 78, false positive alerts have been recorded, the driving behavior identification unit 80 may analyze the alert as a false positive alert. Similarly, if diagnostics subsystem 46 of ADAS 22 provides an alert at a location where historically, as indicated by the stored historical data 78, valid alerts have been recorded, the driving behavior identification unit 80 may analyze the alert as a valid alert. As another example, with respect to a lane departure warning feature of ADAS 22, if diagnostics subsystem 46 of ADAS 22 provides a lane departure alert when a turn signal has been activated by the operator of vehicle 12 (and therefore intending to shift lanes), the driving behavior identification unit 80 may analyze the alert as a false positive alert.

Driving behavior identification unit 80 may analyze or process contextual data generated by subsystems 40, 42, 44, and/or 46 of ADAS 22 to characterize the driving behavior (e.g., speeding, accelerating, braking, lane shifting, weaving patterns, etc.) that caused either activation of the alert (for both false positive and valid alerts) by ADAS 22 or the autonomous action by ADAS 22. The driving behavior identification unit 80 may also receive reaction data generated by subsystems 40, 42, and/or 44 of ADAS 22 to determine speeding, accelerating, braking, lane shifting, and/or weaving patterns of the operator of vehicle 12 in response to the alert generated by subsystem 46 of ADAS 22, and subsequently compare the contextual data with the reaction data. The driving behavior identification unit 80 may identify reaction data as data corresponding to a timestamp tag just after the time (within a pre-determinable threshold) in which the alert was provided to vehicle 12.

A comparison of the identified reaction data with the contextual data may show whether the reaction data is consistent with the contextual data. A consistent correlation between reaction data and contextual data may represent a finding that an operator's driving behavior did not change in response to the alert (alternatively, an inconsistent correlation may indicate that operator's driving behavior did change). Further, a consistent correlation may represent either safe driving behavior, or alternatively, unsafe driving behavior. For instance, if the alert is classified as a false positive alert (e.g., based upon stored historical data 78), a consistent correlation indicates that the operator may have ignored the false positive alert, which may be indicative of safe or risk averse driving behavior. However, if the alert is classified as a valid alert (e.g., based upon stored historical data 78), a consistent correlation may indicate that the valid alert was not followed, which may be indicative of unsafe, risky, and/or unresponsive driving behavior.

Similarly, an inconsistent correlation between reaction data and contextual data may represent a finding that an operator's driving behavior did change in response to the alert. Further, an inconsistent correlation may represent either safe or unsafe driving behavior. For instance, if the alert is classified as a valid alert (e.g., based upon stored historical data 78), an inconsistent correlation indicates that the operator has taken action in response to the valid alert, which may be indicative of safe, risk averse, or responsive driving behavior. However, if the alert is classified as a false positive alert (e.g., based upon stored historical data 78), an inconsistent correlation indicates that the operator may have taken action in response to the false positive alert, which may be indicative of unsafe or risky driving behavior.

As will be discussed below, the comparison results of the identified reaction data with the contextual data, which shows whether the reaction data is consistent with the contextual data, may be stored in memory 72 or may be stored external to computer system 16 (e.g., memory 52 or other memory units), and/or may be utilized by profile generation/update unit 82 to generate or update a driving metric for the operator.

In some embodiments, each of one or more driving behaviors characterized by driving behavior identification unit 80 may be associated with tags or other metadata indicating the circumstances in which the driving behavior occurred. As briefly described above, driving behavior identification unit 80 may identify the contextual data generated by ADAS 22 as the data associated with a tagged first timestamp at the time ADAS 22 activated an alert or autonomous action, and may identify the reaction data generated by subsystems 40, 42, and/or 44 as the data associated with a tagged second timestamp that is just after the first timestamp (within a pre-determinable threshold), for example.

Driving behavior identification unit 80 may use other tags or other metadata associated with the contextual data and reaction data in order to pair them as a set to characterize driving behavior in response to an ADAS alert. For example, driving behavior identification unit 80 may identify contextual data and reaction data that are both associated with an operator's name or other identifier, in addition to the first and second timestamps, to character the particular driver's driving behavior in response to an ADAS alert. As another example, driving behavior identification unit 80 may identify contextual data and reaction data that are both associated with a location, in addition to the first and second timestamps, to characterize the general driving behavior (i.e., not to a particular driver) in response to an ADAS alert at the location.

In alternative embodiments, data processing unit 54, as opposed to driving behavior identification unit 80, may identify some or all of the driving behaviors. In such embodiments, driving behavior identification unit 80 may be excluded from data analysis unit 74, or may operate in conjunction with data processing unit 54. For example, data processing unit 54 may identify some types of driving behaviors, while driving behavior identification unit 80 identifies other types of driving behaviors and/or higher-level driving behaviors. In one such embodiment, for instance, data processing unit 54 may determine tailgating distances to other vehicles using data from external sensor 30 and image recognition algorithms (e.g., to identify an object ahead of vehicle 12 as another vehicle), and driving behavior identification unit 80 may use that information, along with data from third party server 18 or another server, to determine an average tailgating distance for each of a number of different weather conditions (e.g., sunny, partly cloudy, cloudy, fog, rain, snow, icy roads, etc.).

Profile generation/update unit 82 may be generally configured to use the driving behaviors identified by driving behavior identification unit 80 and/or data processing unit 54, to populate and/or update fields of an operator profile for the operator of vehicle 12 in driver profiles database 76. Each of a number of different drivers (including the operator of vehicle 12) may be associated with a different profile in driver profiles database 76, with each profile having one or more fields of information.

Each driver profile may, in some embodiments, include a driving metric that generally indicates how responsible or trustworthy the operator may be (e.g., as determined based upon various driving behaviors and/or other types of information that are probative of responsibility/trustworthiness), and more particularly, how reliant or responsive the operator may be to an ADAS alert. As discussed above, the comparison results of the identified reaction data with the contextual data, which shows whether the reaction data is consistent with the contextual data, may be utilized by profile generation/update unit 82 to generate or update a driving metric for the operator. In some embodiments, each profile may also include a number of fields indicative of demographic and/or personal information (e.g., gender, age, education level, profession, disabilities/impairments/limitations, etc.), vehicle information (e.g., vehicle model, year, and/or color), and/or other information.

The driving metric may change depending upon whether an ADAS alert is a valid alert or a false positive alert. The driving metric may also change depending on whether the reaction data is consistent with the contextual data. Specifically, the driving metric may be lowered when the operator exhibits unsafe or unresponsive driving behavior, such as when (i) the alert is classified as a false positive alert, and the reaction data is not consistent with the contextual data, or when (ii) the alert is classified as a valid alert, and the reaction data is consistent with the contextual data. Similarly, the driving metric may be increased when the operator exhibits safe or responsive driving behavior, such as when (i) the alert is classified as the false positive alert, and the reaction data is consistent with the contextual data, or when (ii) the alert is classified as the valid alert, and the reaction data is not consistent with the contextual data. In some embodiments, the driving metric may be increased when the operator exhibits unsafe or unresponsive driving behavior, and the driving metric may be lowered when the operator exhibits safe or responsive driving behavior.

While FIG. 1 depicts an embodiment in which vehicle telematics data may be generated and transmitted to computer 16 by an on-board system 14, in other embodiments some or all of the vehicle telematics data may instead be generated and/or transmitted to computer 16 by a mobile electronic device (e.g., a smartphone, a wearable electronic device, and/or another mobile electronic device of the operator and/or a passenger of vehicle 12). For example, an accelerometer, gyroscope, compass, and/or camera of the operator's smartphone may be used to generate vehicle telematics data, which may be transmitted to computer 16 by either the smartphone itself or on-board system 14 (e.g., after the smartphone transmits the telematics data to on-board system 14 via a short-range communication technology such as WiFi or Bluetooth). In some embodiments where the operator's or passenger's mobile electronic device transmits some or all of the vehicle telematics data to computer 16, the mobile electronic device may include an interface (e.g., similar to interface 60) that is configured to transmit the data to computer 16 via network 20 or another network.

Exemplary Inputs for Identifying Driving Behavior and Generating or Modifying Driver Profiles

FIG. 2 depicts exemplary data categories 100 that may be utilized by the system 10 of FIG. 1 to identify driving behavior, and/or generate or modify an operator profile, such as an operator profile included in driver profiles database 76 of computer system 16. The data categories 100 may include operational data 102, sensor data 104, diagnostic data 106, location data 110, driver-provided data 112, and/or third party data 114. In other embodiments, the data categories 100 may include more, fewer, and/or different categories of data, and/or each category shown in FIG. 2 may include more, fewer, and/or different types of data.

Operational data 102 may include one or more types of data relating to operation metrics of a vehicle, such as speed data, acceleration data, braking data, weaving data, steering data, ADAS data (e.g., whether/when a particular feature of ADAS is engaged), drive mode data (e.g., data indicating whether the operator selected a “comfort,” “eco” or “sport” mode), headlight data (e.g., whether/when headlights are turned on), turn signal data (e.g., whether/when turn signals are used), and/or windshield wiper data (e.g., whether/when front and/or rear wipers are used). Some or all of operational data 102 may be data generated by subsystems 40, 42, 44, 46 and/or ADAS 22 of FIG. 1, one or more other subsystems of vehicle 12 not shown in FIG. 1, and/or a mobile electronic device of an operator or passenger.

Sensor data 104, which may overlap in definition with operational data 102 to some degree, may include one or more types of data indicative of internal and external conditions of a vehicle, and particularly conditions that may be captured by cameras, weight sensors, and/or other types of sensors. Sensor data 104 may include, for example, traffic condition data, weather condition data, data indicating the number of passengers in the vehicle, data indicating when particular seatbelts are used, data indicative of tire pressure, and/or driver image data. Some or all of sensor data 102 may be data generated by external sensor 30, external sensor 32, and/or internal sensor(s) 38 of FIG. 1, and/or by a mobile electronic device of an operator or passenger. For example, external sensor 30 and/or external sensor 32 may generate the traffic and/or weather data, internal sensor(s) 38 may generate the data indicative of number of passengers, seatbelt usage data, and/or driver image data, and a different sensor of vehicle 12 (not shown in FIG. 1) may generate the tire pressure data.

Diagnostic data 106 may include one or more diagnostic status codes indicative of the state of hardware and/or software systems of a vehicle, such as data indicative of safety alerts from various sensors (e.g., a check engine alert, a low tire pressure warning, an oil change reminder, etc.), data indicative of whether particular feature(s) of ADAS 22 are operational or malfunctioning, data indicative of alerts from ADAS 22, including timestamps associated with the alerts, and/or data indicative of the current version of one or more units of software installed in the vehicle (e.g., for on-board system 14 of FIG. 1). Some or all of diagnostic data 106 may be data generated by diagnostic subsystem 46 of FIG. 1, for example.

Location data 110 may include one or more types of data indicative of vehicle location. For example, location data 110 may include location data obtained from a GPS unit installed in a vehicle (e.g., GPS subsystem 48 of FIG. 1), and/or location data obtained from a mobile device (e.g., smartphone, smart watch, etc.) of the operator that includes a GPS unit.

Driver-provided data 112 may include one or more types of data specific to the operator and his or her vehicle, such as driver age, driver gender, driver education level, driver profession, driver limitations, vehicle model, vehicle year, and/or vehicle color. As the label suggests, driver-provided data 112 may be data that the operator provided (e.g., when filling out an application or other form or questionnaire). Alternatively, some or all of driver-provider data 112 may be obtained in a different manner (e.g., provided by a third party, similar to third party data 114).

Third party data 114 may include one or more types of data sourced by one or more third party entities. For example, third party data 114 may include data indicative of specific driver limitations (e.g., vision impairment, motor skill impairment, etc.), which may be obtained from a governmental entity or other entity. As another example, third party data 114 may include data indicative of traffic conditions, speed limits, and/or road conditions, which may be obtained from a governmental entity, an entity that provides a mapping service, or another entity.

As noted above, the data in the exemplary data categories 100 may be analyzed by the system 10 of FIG. 1 to identify driving behavior, and/or generate or modify an operator profile, such as an operator profile included in driver profiles database 76 of computer system 16. Various examples of how data shown in FIG. 2 may be used to determine patterns in driving behavior, such as risk averse driving behavior, and/or determine/set profile information are provided below in connection with FIG. 3.

Exemplary Information Determined for Driver Profiles

FIG. 3 depicts exemplary profile information categories 150 that may be determined by the system 10 of FIG. 1 (e.g., by data analysis unit 74) when identifying driving behavior patterns and/or generating or modifying an operator profile, such as an operator profile included in driver profiles database 76 of computer system 16. The profile information categories 150 may include driving behavior information 152, feature usage information 154, alert responsiveness information 156, and/or driver state information 158. In other embodiments, the profile information categories 150 may include more, fewer, and/or different categories than are shown in FIG. 3, and/or each category may include more, fewer, and/or different types of information than are shown in FIG. 3.

Driving behavior information 152 may include acceleration patterns, braking patterns, weaving patterns, ADAS usage patterns, compliance with speed limits, and/or compliance with driver limitations. For example, driving behaviors identification unit 80 or another unit of data analysis unit 74 may determine acceleration, braking, weaving patterns, and ADAS usage patterns by analyzing acceleration, braking, weaving data, and ADAS data from operational data 102 of FIG. 2 over a period of time, and/or analyzing historical data 78.

As another example, driving behaviors identification unit 80 or another unit of data analysis unit 74 may determine compliance with speed limits (e.g, a number of times in a particular time period that the posted speed limit is exceeded by more than 5 miles per hour, a maximum amount or percentage by which speed deviates below or above a posted speed limit in a particular time period, etc.) using speed data from operational data 102 (or using acceleration data of operational data 102 to determine speed), and speed limit data from third party data 114, of FIG. 2.

Driving behaviors identification unit 80 or another unit of data analysis unit 74 may determine compliance with driver-specific limitations using one or more types of data within operational data 102 of FIG. 2, one or more types of data within sensor data 104 of FIG. 2, and driver limitation data from third party data 114 of FIG. 2. For example, driving behaviors identification unit 80 or another unit of data analysis unit 74 may use data from external sensor 30 and speed subsystem 42 of FIG. 1 to determine one or more metrics indicating how closely an operator follows other vehicles (“tailgates”) in relation to his or her speed due to an operator-specific limitation. Driving behaviors identification unit 80 or another unit of data analysis unit 74 may determine from the third party server 18 of FIG. 1 (e.g., a department of transportation within a state) or from the operator-provided data (e.g., driver limitations data) that he or she has a particular level of vision impairment (e.g., shortsightedness or poor night vision), motor skill impairment (e.g, due to a handicap or injury), and/or other medical conditions and/or physical characteristics that may limit how he or she can safely operate a vehicle. Thereafter, driving behaviors identification unit 80 may determine whether the operator tends to follow other vehicles at a “safe” distance, in light of known correlations stored in memory (e.g., memory 72) between drivers with similar limitations and the occurrence of accidents. Other driving behaviors (e.g., braking patterns, weaving patterns, windshield wiper usage, etc.) may also, or instead, be analyzed in connection with any driver-specific limitations.

In some implementations, one or more of the types of information in driving behavior information 152 may be further subdivided based upon various conditions (e.g., traffic, weather conditions) in order to determine whether driving behavior changes based upon changing conditions. Specifically, driving behaviors identification unit 80 or another unit of data analysis unit 74 may correlate the sensor data 104 of FIG. 2 to the speed, acceleration, braking, weaving, ADAS data and other data from operational data 102 of FIG. 2 over a period of time. For example, some or all of the driving behavior information 152 may be associated with different weather conditions, different traffic conditions, different times and/or lighting conditions (e.g., day, evening, night, etc.), different vehicle models (e.g., if profile information is separately determined for multiple vehicles of an operator), different types of roads/areas, different enabled ADAS features, and so on. Thus, for instance, driving behavior information 152 may include compliance with speed limits (e.g., an indication of how often and/or long the operator exceeds the speed limit by a pre-determined threshold amount) for heavy traffic, and also compliance with speed limits for light and/or moderate traffic.

As another example, driving behavior information 152 may include compliance with speed limits in school zones, as well as compliance with speed limits on interstate roads. As yet another example, driving behavior information 152 may include acceleration, braking, weaving, and ADAS usage patterns in clear weather, rainy, foggy, and/or snowy/icy weather, heavy traffic, light traffic, etc.

In some embodiments, driving behaviors identification unit 80 or another unit of data analysis unit 74 may correlate the sensor data 104 of FIG. 2 to the speed, acceleration, braking, weaving, ADAS data and other data from operational data 102 of FIG. 2, as well as to valid and false positive ADAS alerts determined by data analysis unit 74, over a period of time. For example, upon analysis of the vehicle data (e.g., sensor data 104 and operational data 102) from vehicle 12 via interface 60 and correlation of such data to valid and false positive ADAS alerts determined by data analysis unit 74, if the driving behavior identification unit 80 determines that a first set of acceleration, braking, and/or weaving data (e.g, from operational data 102) of the operator of vehicle 12 associated with heavy traffic (e.g., sensor data 104) shows a change in data when compared to vehicle data (e.g., from operational data 102, such as data generated by subsystems 40, 42, 44, and/or 46 of ADAS 22) that caused a valid ADAS alert to be generated at the vehicle 12, driving behavior identification unit 80 may determine that the operator exhibited safe driving behavior by abiding by the valid ADAS alert.

If the driving behavior identification unit 80 determines that a second set of acceleration, braking, and/or weaving data (e.g. from operational data 102) of the operator of vehicle 12 associated with light traffic (e.g., sensor data 104) does not show a change in data when compared to vehicle data (e.g., from operational data 102, such as data generated by subsystems 40, 42, 44, and/or 46 of ADAS 22) that caused a valid ADAS alert to be generated at the vehicle 12, driving behavior identification unit 80 may determine that the operator exhibited dangerous driving behavior by ignoring a valid ADAS alert. Based upon a comparison of driving behaviors associated with the first and second sets, driving behavior identification unit 80 may also determine that the operator exhibits safer driving behavior when in heavier traffic.

Feature usage information 154 may include forward collision warning feature usage, a blind spot indication feature usage, a cruise control feature usage, a lane departure warning feature usage, automatic high beam usage, and/or other ADAS feature usage, and may also be indicative of how often the operator uses one or more of the aforementioned features. The aforementioned usages may be road condition, weather and/or time-of-day dependent. For example, driving behaviors identification unit 80 or another unit of data analysis unit 74 may use ADAS data or other data from operational data 102, traffic condition data from sensor data 104, weather condition data from sensor data 104, and/or third party data 114 (e.g., road conditions data) to determine how often (and/or for how long) the operator uses any one or more features of ADAS 22 in various different road, traffic, and/or weather conditions. For illustrative purposes, feature usage information 154 is shown as its own distinct profile information category. However, feature usage information 154 may also be a subset or subdivision of driving behavior 152, namely, ADAS usage patterns.

ADAS alert responsiveness information 156 may include data corresponding to operator responsiveness to both false positive and valid ADAS alerts (e.g., forward collision warning, a blind spot indication warning, a cruise control warning, a lane departure warning, an automatic high beam warning, etc.). Particularly, alert responsiveness information 156 may include and/or utilize data such as diagnostic data 106 (e.g. data indicative of whether particular feature(s) of ADAS 22 are operational or malfunctioning, data indicative of whether alerts from ADAS 22 were indicated to the operator and at what time, data indicative of whether the alerts from ADAS 22 were deactivated and at what time), and/or operational data 102 (e.g., data indicative of whether ADAS has been engaged in the vehicle, data indicative of operational data 102 that caused activation of the alerts from ADAS 22, data indicative of operational data 102 that caused deactivation of the alerts from ADAS 22) and possibly data from a third party such as data from third party server 18 of FIG. 1, to determine operator responsiveness to ADAS alerts.

For example, as shown in FIG. 4, driving behaviors identification unit 80 or another unit of data analysis unit 74 may determine that the operator followed a valid ADAS alert, as described in scenario 160. After determining that data 162 suggests that a valid ADAS alert was indicated to the operator, driving behaviors identification unit 80 or another unit of data analysis unit 74 may determine that the ADAS alert has been deactivated (164) at a timestamp (168) subsequent to the timestamp of data 162 (e.g., the ADAS alert has been deactivated after a pre-determinable amount of time that has elapsed since the timestamp of data 162). Alternatively, driving behaviors identification unit 80 or another unit of data analysis unit 74 may determine that the operational data 102 that caused the ADAS alert to be indicated to the operator in the first place changed (166), or did not remain consistent, at a timestamp (168) subsequent to the timestamp of data 162 (e.g., the operational data 102 changed after a pre-determinable amount of time that has elapsed since the timestamp of data 162). As will be further described below, data analysis unit 74, by comparing data 162 with data 164, 166, and 168 of scenario 160, may increase a driving metric of the operator.

Similarly, as shown in FIG. 4, driving behaviors identification unit 80 or another unit of data analysis unit 74 may determine that the operator ignored a valid ADAS alert, as described in scenario 170. After determining that data 172 suggests that a valid ADAS alert was indicated to the operator, driving behaviors identification unit 80 or another unit of data analysis unit 74 may determine that the ADAS alert still remains activated (174) at a timestamp (178) subsequent to the timestamp of data 172 (e.g., the ADAS alert still remains activated after a pre-determinable amount of time that has elapsed since the timestamp of data 172). Alternatively, driving behaviors identification unit 80 or another unit of data analysis unit 74 may determine that the operational data 102 that caused the ADAS alert to be indicated to the operator in the first place did not change (176), or remained consistent, at a timestamp (178) subsequent to the timestamp of data 172 (e.g., the operational data 102 did not change after a pre-determinable amount of time that has elapsed since the timestamp of data 172). As will be further described below, data analysis unit 74, by comparing data 172 with data 174, 176, and 178 of scenario 170, may lower a driving metric of the operator.

As another example, as shown in FIG. 5, driving behaviors identification unit 80 or another unit of data analysis unit 74 may determine that the operator followed a false positive ADAS alert, as described in scenario 180. After determining that data 182 suggests that a false positive ADAS alert was indicated to the operator, driving behaviors identification unit 80 or another unit of data analysis unit 74 may determine that the ADAS alert has been deactivated (184) at a timestamp (188) subsequent to the timestamp of data 182 (e.g., the ADAS alert has been deactivated after a pre-determinable amount of time that has elapsed since the timestamp of data 182). Alternatively, driving behaviors identification unit 80 or another unit of data analysis unit 74 may determine that the operational data 102 that caused the ADAS alert to be indicated to the operator in the first place changed (186), or did not remain consistent, at a timestamp (188) subsequent to the timestamp of data 182 (e.g., the operational data 102 changed after a pre-determinable amount of time that has elapsed since the timestamp of data 172). As will be further described below, data analysis unit 74, by comparing data 182 with data 184, 186, and 188 of scenario 180, may lower a driving metric of the operator.

Similarly, as shown in FIG. 5, driving behaviors identification unit 80 or another unit of data analysis unit 74 may determine that the operator ignored a false positive ADAS alert, as described in scenario 190. After determining that data 192 suggests that a false positive ADAS alert was indicated to the operator, driving behaviors identification unit 80 or another unit of data analysis unit 74 may determine that the ADAS alert still remains activated (194) at a timestamp (198) subsequent to the timestamp of data 192 (e.g., the ADAS alert still remains activated after a pre-determinable amount of time that has elapsed since the timestamp of data 192). Alternatively, driving behaviors identification unit 80 or another unit of data analysis unit 74 may determine that the operational data 102 that caused the ADAS alert to be indicated to the operator in the first place did not change (196), or remained consistent, at a timestamp (198) subsequent to the timestamp of data 192 (e.g., the operational data 102 did not change after a pre-determinable amount of time that has elapsed since the timestamp of data 192). As will be further described below, data analysis unit 74, by comparing data 192 with data 194, 196, and 198 of scenario 190, may increase a driving metric of the operator.

Referring back to FIG. 3, driver state information 158 may include driver attentiveness and/or driver emotional state. For example, driving behaviors identification unit 80 or another unit of data analysis unit 74 may determine how attentive an operator is (e.g., gaze direction, how often he or she checks instruments and/or the rearview mirror, how often he or she checks an ADAS alert, etc.) by using image recognition and/or other image processing techniques to process driver image data from sensor data 104 of FIG. 2. As another example, driving behaviors identification unit 80 or another unit of data analysis unit 74 may determine an operator's emotional state and/or level of attentiveness (e.g., calm, angry, distracted, etc.) by using image recognition and/or other image processing techniques to process driver image data from sensor data 104. In some embodiments, driving behaviors identification unit 80 or another unit of data analysis unit 74 may correlate the operator state information 158 with the sensor data 104 and operational data 102 of FIG. 2, as well as to valid and false positive ADAS alerts determined by data analysis unit 74, over a period of time, to identify driving behavior and/or generate/modify driver profiles.

Some or all of the types of profile information discussed above, and/or other types of information, may be used (e.g., by profile generation/update unit 82 of FIG. 1) to populate and/or update one or more profile fields of a profile for a particular driver. For example, profile generation/update unit 82 may use some or all types of profile information within driving behavior information 152, feature usage information 154, alert responsiveness information 156, and/or driver state information 158 as profile field values, and/or also calculate a driving metric based upon some or all types of driving behavior information 152, feature usage information 154, alert responsiveness information 156, and/or driver state information 158. Therefore, such a profile may provide various driving behaviors of an operator, such as the operator's preferred ADAS features, responsiveness to the preferred ADAS features, etc.

When calculating the driving metric, various profile information types and/or categories may be more heavily weighted than others. For example, responsiveness to ADAS alerts may be weighted more heavily than weather-specific windshield wiper usage. Generally, specific types of profile information may be used to determine the driving metric if it is known a priori (e.g., from past correlations with driver actions) or believed that those types of information are probative of how trustworthy or responsible the operator is.

The driving metric may be determined using various types of information shown in FIG. 3, and also using information about the operator and/or vehicle (e.g., data included in driver-provided data 112 of FIG. 2, such as age, gender, education level, profession, vehicle model, etc.). As just one more specific example, the driving metric or other information in an operator profile may be based at least in part upon a joint consideration of (1) a color of the vehicle, (2) times of day when the vehicle is driven, (3) weather in which the vehicle is driven (e.g., in view of the assumption or known correlation that vehicles of certain colors may be less visible at certain times of day and/or in certain types of weather), and (4) ADAS alert responsiveness (e.g., from information 156).

Exemplary Use Cases for Driver Profiles

Once an operator profile is determined (e.g., generated or updated using some or all of the profile information shown in FIG. 2, FIG. 3, and/or other types of information), one or more fields of the profile may be used in any of a number of different ways, depending upon the embodiment.

In some embodiments, the operator profile may be used in connection with driver education and/or licensing. For example, situation-specific driving behaviors reflected in the profile (e.g., driving behavior in specific types of weather and/or traffic) may be used by a government entity for licensing or re-licensing of drivers. As another example, driver profiles may be used to rate how well or responsibly a driving instructor drives, and/or how well or responsibly his or her students drive (with the latter ratings potentially also being used to rate the instructor). In embodiments such as these, driver profile information may be transmitted to a remote computing system (e.g., third party server 18 of FIG. 1) for display to one or more individuals positioned to act upon the information (e.g., to approve the grant of a license, or provide a performance review to an instructor, etc.).

In still other embodiments, driver profiles may be used to adjust costs for usage-based insurance and/or other insurance premiums. For example, an underwriting department of an insurer may use driver profile information to gauge risk and set appropriate premiums. Alternatively, the costs of usage-based insurance may be automatically calculated by a computing system (e.g., computer system 16 of FIG. 1) based upon the operator profile information.

In still other embodiments, driver profiles may be used to influence resale values of vehicles. In particular, driver profile information indicative of how aggressively or conservatively the operator drove the vehicle may cause the value to go down or up, respectively. In certain embodiments such as these, driver profile information may be transmitted to a remote computing system (e.g., third party server 18 of FIG. 1, or a personal computing device of a potential buyer, etc.) for display to one or more individuals positioned to act upon the information (e.g., set the vehicle resale price, or buy the vehicle).

In still other embodiments, driver profiles may be used by fleet owners to provide rental vehicle discounts. For example, driver profile information may be transmitted to a remote computing system (e.g., third party server 18 of FIG. 1) for display to one or more individuals positioned to act upon the information (e.g., an agent who can apply the discount), or to cause the discount to be automatically applied to a rental fee.

In still other embodiments, driver profiles may be used by car sharing services to provide discounts. For example, driver profile information may be transmitted to a remote computing system (e.g., third party server 18 of FIG. 1) for display to one or more individuals positioned to act upon the information (e.g., an agent who can apply the discount), or to cause the discount to be automatically applied to a car share fee.

In still other embodiments, driver profiles may be used for other purposes, such as determining how a particular individual would likely care for, maintain, or be compatible with driving a vehicle (e.g., a rental vehicle with or without ADAS features, autonomous vehicle, etc.), estimating how long vehicle components (e.g., tires, brake pads, rotors, etc.) will last, and so on.

In some embodiments where driver profiles include driving metrics (as discussed above), such scores may be used in a number of different situations where the operator's trustworthiness or driving behavior is important. For example, the driving metric may be used by an insurance entity to adjust a price to risk model associated with the operator based upon the operator's driving metric. In embodiments such as these, driving metrics may be transmitted to a remote computing system (e.g., third party server 18 of FIG. 1) for display to one or more individuals positioned to act upon the information (e.g., authorize price to risk model adjustment), and/or for automated adjustment of the price to risk model.

As another example, the driving metric may be used by a credit rating entity to raise or lower the operator's credit score. In embodiments such as these, driving metrics may be transmitted to a remote computing system (e.g., third party server 18 of FIG. 1) for display to one or more individuals positioned to act upon the information (e.g., authorize a credit score change), and/or for automated adjustment of the credit score.

As another example, the driving metric may be submitted to an employer in connection with a resume and/or application for a particular job. A driving metric may be especially pertinent to jobs that involve frequent driving, such as an operator for restaurant delivery, a ride-sharing driver, etc. In embodiments such as these, a driving metric may be transmitted to a remote computing system (e.g., third party server 18 of FIG. 1) for display to one or more individuals positioned to act upon the information (e.g., hire the individual associated with the driving metric).

As yet another example, the driving metric may be used to enable “IOUs” with particular service providers (e.g., a taxi service, ride-sharing service, etc.). In embodiments such as these, driving metrics of driver profiles may be transmitted to a remote computing system (e.g., third party server 18 of FIG. 1), after which the computing system may indicate to one or more agents of the service provider that an IOU may be accepted from the individual.

In another embodiment, driving metrics need not be transmitted to a remote computing system. The data analysis unit 74 or data processing unit 54 themselves may adjust the price to risk model, credit rating, insurance rating, review, permanent credit, or temporary credit associated with the operator based upon at least the portion of the operator profile.

Exemplary Computer-Implemented Method for Detecting and Acting Upon Responsiveness to Vehicle Alerts

FIG. 6 is a flow diagram of an exemplary computer-implemented method 300 for detecting and acting upon operator responsiveness to alerts provided by ADAS 22. The method 300 may be implemented by data analysis unit 74 or data processing unit 54 of FIG. 1, for example.

In the method 300, data analysis unit 74 or data processing unit 54 may determine that an on-board system of a vehicle (e.g., on-board system 14 of vehicle 12 in FIG. 1) has generated an alert of an ADAS (e.g., ADAS 22) for a driving activity (block 302). For example, it may be determined that the on-board system 14, via ADAS 22, indicated a forward collision warning feature, a blind spot indication feature, a cruise control feature, a lane departure warning feature, an automatic high beam feature, and/or other advanced driver assistance features. Data analysis unit 74 or data processing unit 54 may determine whether alerts from ADAS 22 are false positive alerts, or alternatively valid alerts, by analyzing data from vehicle 12 (e.g., the data received from on-board system 14 via interface 60) and a number of other vehicles stored in historical driving data database 78.

The method 300 may then log a first set of measurements data associated with the alert of the ADAS (e.g., ADAS 22) in a database (e.g., historical driving data database 78) saved in a memory (e.g., memory 72) (block 304). The first set of measurements may include vehicle data that caused activation of the alert of ADAS 22 that was generated by the on-board system of the vehicle (e.g., by diagnostic subsystem 46 of on-board system 14 in FIG. 1). The vehicle data may include one or more operational data 102, sensor data 104, diagnostic data 106, location data 110, driver-provided data 112, and/or third party data 114 of FIG. 2, and may further include a plurality of time stamps associated with one or more of the vehicle data, which may be indicative of the time as to when the on-board system of the vehicle has indicated the alert to the operator. The timestamps may be determined as a specific time or a time range (e.g., on or before a particular date), for example.

In one embodiment where the method 300 is implemented by a server remote from the vehicle (e.g., a server of computer system 16 of FIG. 1), the vehicle data may be received at the server, from the on-board system, via a wireless link (e.g., via network 20 of FIG. 1). For example, if method 300 has determined that a lane departure alert provided by ADAS 22 has been generated at the vehicle 12, method 300 may log a first set of measurement data associated with the lane departure alert, such as operational data 102 (e.g., whether a turn signal has been activated at the time the lane departure alert was provided by ADAS 22, steering data), sensor data 104 (e.g., driver images of whether a vehicle is straddling a lane at the time the lane departure alert was provided by ADAS 22), diagnostic data 106 (e.g., whether the software version of the lane departure warning feature is up to date at the time the lane departure alert was provided by ADAS 22), location data 110 (e.g., where the vehicle was located at the time the lane departure alert was provided by ADAS 22), driver-provided data 112 (e.g., vehicle model), and/or third party data 114 (e.g., road conditions which may indicate whether the lane has been clearly indicated with visible lane markings) in a database saved in memory (e.g., at historical database 78).

The first set of measurement data not only may serve to describe characteristics of the driving activity at the time an ADAS alert has been provided, but may also may serve to assist in classifying whether an alert is a false positive alert, or alternatively a valid alert. For example, if the operator images show that a vehicle is not leaving a lane at the time the lane departure alert was provided by ADAS 22, or if the operational data 102 shows that a turn signal has been activated at the time the lane departure alert was provided by ADAS 22, the data analysis unit 74 or data processing unit 54 may determine that the alerts from ADAS 22 are false positive alerts.

The method 300 may then receive a second set of measurements data in response to the alert for the driving activity (block 306). The second set of measurements may include vehicle data in response to the alert of ADAS 22 that was generated by the on-board system of the vehicle. The vehicle data may include one or more operational data 102, sensor data 104, diagnostic data 106, location data 110, driver-provided data 112, and/or third party data 114 of FIG. 2, and may further include a timestamp and/or location associated with one or more of the vehicle data. For example, in response to the lane departure alert provided by ADAS 22, method 300 may receive a second set of measurement data associated with the lane departure alert, such as operational data 102 (e.g., steering data) and sensor data 104 (e.g., driver images of whether a vehicle is straddling a lane).

Method 300 in block 306 may also detect and/or record an amount of time that has elapsed between the first set of measurements data and the second set of measurements data. Data analysis unit 74 or data processing unit 54 may be configured with a pre-determinable time threshold to analyze any pairings of the first and second sets of measurements data that have an elapsed time between them that is equal to or less than the pre-determinable time threshold, and may ignore any pairings of the first and second sets of measurements data that have an elapsed time between them that is greater than the pre-determinable time threshold. For example, data analysis unit 74 or data processing unit 54 may analyze the pairing of the first set of measurement data associated with the lane departure alert and the second set of measurement data in response to the lane departure alert if an elapsed time between them (i.e., timestamps associated with the first and second sets of measurement data) is equal to or less than a pre-determinable time threshold, or may ignore otherwise.

The method 300 may then compare the analyzed first set of measurements data with the second set of measurements data to determine whether the second set of measurements data is consistent with the first set of measurements data (block 308).

A consistent correlation between the first set of measurements data and second set of measurements data may represent a finding that an operator's driving behavior did not change in response to the alert. For example, and continuing with the lane departure alert example, data analysis unit 74 or data processing unit 54 may determine that the first set of measurements data (e.g., steering data and driver images of a vehicle straddling a lane) and the second set of measurements data (e.g., steering data and driver images of a vehicle straddling a lane) are the same, or in other words, consistent. Such a finding may represent that upon being alerted of a lane departure alert, the operator neither changed his or her steering of the vehicle, nor adjusted the vehicle to be within a lane, and thus was unresponsive to a lane departure alert.

A consistent correlation may represent either safe driving behavior or unresponsive driving behavior, however. For instance, if data analysis unit 74 or data processing unit 54 determines that the lane departure alert from ADAS 22 is a false positive alert (e.g., the first set of measurements data indicates that a turn signal has been activated at the time the lane departure alert was provided by ADAS 22, thereby showing the operator intended to change lanes), a consistent correlation between the steering data and driver images of the first and second sets of measurements data may indicate that the operator may have ignored the false positive alert, which may be indicative of safe driving behavior.

However, if data analysis unit 74 or data processing unit 54 classifies the lane departure alert as a valid alert (e.g., the first set of measurements data indicates that a turn signal has not been activated at the time the lane departure alert was provided by ADAS 22, thereby showing no intention for the operator to change lanes), a consistent correlation between the steering data and driver images of the first and second sets of measurements data indicates that the operator may have ignored the valid alert, which may be indicative of unsafe or unresponsive driving behavior.

Similarly, an inconsistent correlation between the first set of measurements data and second set of measurements data may represent a finding that an operator's driving behavior did change in response to the alert. For example, and continuing with the lane departure alert example, data analysis unit 74 or data processing unit 54 may determine that the first set of measurements data (e.g., steering data and driver images of a vehicle straddling a lane) and the second set of measurements data (e.g., steering data and driver images of a vehicle straddling a lane) are different, or in other words, inconsistent. Such a finding may represent that upon being alerted of a lane departure alert, the operator changed his or her steering of the vehicle, or adjusted the vehicle to be within a lane, and thus may have followed, or responded to, the lane departure alert.

Further, an inconsistent correlation may represent either safe driving behavior or alternatively, unresponsive driving behavior. For instance, if data analysis unit 74 or data processing unit 54 determines that the lane departure alert from ADAS 22 is a valid alert (e.g., the first set of measurements data indicates that a turn signal has not been activated at the time the lane departure alert was provided by ADAS 22, thereby showing no intention for the operator to change lanes), an inconsistent correlation between the steering data and driver images of the first and second sets of measurements data indicates that the operator has taken action in response to the valid alert, which may be indicative of safe driving behavior.

However, if data analysis unit 74 or data processing unit 54 determines that the lane departure alert from ADAS 22 is a false positive alert (e.g., based upon stored historical data 78, there are unclear lane markings at the location where the lane departure alert was activated), an inconsistent correlation between the steering data of the first and second sets of measurements data indicates that the operator may have taken action indicative of unsafe or risky driving behavior in response to the false positive alert, such as causing or nearly causing a vehicle collision with an adjacent vehicle after adjusting the vehicle according to the lane departure warning feature of ADAS 22 that responds to unclear lane markings.

Based upon the comparison at block 308, a driving metric may be determined for an operator of the vehicle (block 310). The driving metric may be lowered when the operator exhibits unsafe or risky driving behavior, such as when (i) the alert is classified as a false positive alert based upon the first set of measurements data and the portion of the second set of measurements data is not consistent with the portion of the first set of measurements data; (ii) the alert is classified as a false positive alert based upon the first set of measurements data, and the alert no longer remains indicated to the operator of the vehicle based upon the second set of measurements data; (iii) the alert is classified as a valid alert based upon the first set of measurements data, and the portion of the second set of measurements data is consistent with the portion of the first set of measurements data; or (iv) the alert is classified as a valid alert based upon the first set of measurements data, and the alert remains indicated to the operator of the vehicle based upon the second set of measurements data.

The driving metric may be increased when the operator exhibits safe or risk averse driving behavior, such as when (i) the alert is classified as the false positive alert based upon the first set of measurements data, and the portion of the second set of measurements data is consistent with the portion of the first set of measurements data; (ii) the alert is classified as the false positive alert based upon the first set of measurements data, and the alert remains indicated to the operator of the vehicle based upon the second set of measurements data; (iii) the alert is classified as the valid alert based upon the first set of measurements data, and the portion of the second set of measurements data is not consistent, or inconsistent, with the portion of the first set of measurements data; or (iv) the alert is classified as the valid alert based upon the first set of measurements data, and the alert no longer remains indicated to the operator of the vehicle based upon the second set of measurements data.

The operator profile associated with the operator (e.g., in driver profiles database 76 of FIG. 1) may be set or adjusted to reflect the driving metric associated with a particular driving activity (block 312). Block 312 may correspond to the generation of a new driver profile and/or new driver profile fields, or to the update of an existing driver profile.

In some embodiments, the driving metric may be increased when the operator exhibits unsafe or risky driving behavior, and the driving metric may be lowered when the operator exhibits safe or risk averse driving behavior.

In some embodiments, the operator profile associated with the operator may be set or adjusted to reflect additional information associated with the driving activity that is not necessarily associated with the alert of the ADAS, in addition to the comparison as depicted in block 310. For example, although not shown, method 300 may receive measurements data such as sensor data 104 (e.g., data indicating the number of passengers in the vehicle), driver-provided data 112 (e.g., age, gender, education level, profession), and/or third party data 114 (e.g., driver limitations), and the operator profile associated with the operator may be set or adjusted to reflect such data. For example, it may be determined that an operator of a vehicle has one or more limitations specific to a medical or physical condition, such as impaired vision (e.g., shortsightedness or poor night vision), that the operator has impaired motor skills (e.g., causing slow reaction times), that the operator is driving alone or has passengers in the vehicle, etc. Method 300 may receive such data after requesting one or more records from a remote server via a network (e.g., from third party server 18 of FIG. 1 via network 20), for example.

Although not shown, method 300 may further include transmitting, to a particular entity, at least the portion of the operator profile that was set at block 312, e.g., by generating an instruction to transmit the profile or a profile portion. The entity may be an entity (e.g., insurance institution) that improves a price to risk model associated with the operator based upon the profile or profile portion, an entity (e.g., financial institution) that improves a credit rating associated with the operator based upon the profile or profile portion, an entity (e.g., an insurer) that improves an insurance rating associated with the operator based upon the profile or profile portion, an entity (e.g., an employer) that reviews the profile or profile portion in connection with a job sought by the operator, or an entity (e.g., a rental vehicle company, taxi service, etc.) that offers a permanent or temporary credit (e.g., a discount or IOU), in connection with a good or service offered by the entity based upon the profile or profile portion, for example.

Generally, such ratings may improve in response to a consistent correlation representative of safe or risk averse driving behavior and/or an inconsistent correlation representative of safe or risk averse driving behavior, as discussed above. Similarly, such ratings may worsen in response to a consistent correlation representative of unsafe or risky driving behavior and/or an inconsistent correlation representative of unsafe or risky driving behavior, as discussed above. Such ratings may also adjust based upon additional information associated with a particular driving activity that is not necessarily associated with the alert of the ADAS, such as driver-provided data 112 (e.g., age, gender, education level, profession) and/or third party data 114 (e.g., driver limitations).

The method 300 may include additional, less, or alternate actions, including those discussed elsewhere herein.

In one aspect, a computer-implemented method for detecting and acting upon operator responsiveness to vehicle alerts may be provided. The method may include, via one or more processors, transceivers, servers, and/or sensors: (1) determining that an Advanced Driver Assistance System (ADAS) of a vehicle generated an alert for a driving activity; (2) receiving a set of measurements data associated with the alert of the ADAS; (3) comparing the set of measurements data with a baseline of measurements data; (4) determining a driving metric for an operator of the vehicle based upon the comparing; and/or (5) setting at least a portion of an operator profile associated with the operator with the driving metric. The method may include additional, less, or alternate actions, including those discussed elsewhere herein.

In another aspect, a computer system configured to detect and/or act upon operator responsiveness to vehicle alerts may be provided. The system may include one or more processors, transceivers, servers, and/or sensors configured to: (1) determine that an Advanced Driver Assistance System (ADAS) of a vehicle generated an alert for a driving activity; (2) receive a set of measurements data associated with the alert of the ADAS; (3) compare the set of measurements data with a baseline of measurements data: (4) determine a driving metric for an operator of the vehicle based upon the comparing; and/or (5) set at least a portion of an operator profile associated with the operator with the driving metric. The system may include additional, less or alternate functionality, including that discussed elsewhere herein.

Exemplary Computer System for Generating and/or Using Driver Profiles

FIG. 7 is a block diagram of an exemplary computer system 500 on which a computer-implemented method may operate in accordance with any of the embodiments described above. The computer system 500 of FIG. 7 includes a computing device in the form of a computer 510. Components of the computer 510 may include, but are not limited to, a processing unit 520, a system memory 530, and a system bus 521 that couples various system components including the system memory to the processing unit 520. The system bus 521 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include the Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus (also known as Mezzanine bus).

Computer 510 typically includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by computer 510 and includes both volatile and nonvolatile media, and both removable and non-removable media. By way of example, and not limitation, computer-readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, read only memory (ROM), EEPROM, FLASH memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by computer 510. Communication media typically embodies computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), infrared and other wireless media. Combinations of any of the above are also included within the scope of computer-readable media.

The system memory 530 includes computer storage media in the form of volatile and/or nonvolatile memory such as ROM 531 and RAM 532. A basic input/output system (BIOS) 533, containing the basic routines that help to transfer information between elements within computer 510, such as during start-up, is typically stored in ROM 531. RAM 532 typically contains data and/or program modules that are immediately accessible to, and/or presently being operated on by, processing unit 520. By way of example, and not limitation, FIG. 7 illustrates operating system 534, application programs 535, other program modules 536, and program data 537.

The computer 510 may also include other removable/non-removable, volatile/nonvolatile computer storage media. By way of example only, FIG. 7 illustrates a hard disk drive 541 that reads from or writes to non-removable, nonvolatile magnetic media, a magnetic disk drive 551 that reads from or writes to a removable, nonvolatile magnetic disk 552, and an optical disk drive 555 that reads from or writes to a removable, nonvolatile optical disk 556 such as a CD ROM or other optical media. Other removable/non-removable, volatile/nonvolatile computer storage media that can be used in the exemplary operating environment include, but are not limited to, magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tape, solid state RAM, solid state ROM, and the like. The hard disk drive 541 is typically connected to the system bus 521 through a non-removable memory interface such as interface 540, and magnetic disk drive 551 and optical disk drive 555 are typically connected to the system bus 521 by a removable memory interface, such as interface 550.

The drives and their associated computer storage media discussed above and illustrated in FIG. 7 provide storage of computer-readable instructions, data structures, program modules and other data for the computer 510. In FIG. 7, for example, hard disk drive 541 is illustrated as storing operating system 544, application programs 545, other program modules 546, and program data 547. Note that these components can either be the same as or different from operating system 534, application programs 535, other program modules 536, and program data 537. Operating system 544, application programs 545, other program modules 546, and program data 547 are given different reference numbers in FIG. 7 to illustrate that, at a minimum, they are different copies. A user may enter commands and information into the computer 510 through input devices such as a keyboard 562 and cursor control device 561, commonly referred to as a mouse, trackball or touch pad. A monitor 591 or other type of display device is also connected to the system bus 521 via an interface, such as a graphics controller 590. In addition to the monitor 591, computers may also include other peripheral output devices such as printer 596, which may be connected through an output peripheral interface 595.

The computer 510 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 580. The remote computer 580 may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer 510, although only a memory storage device 581 has been illustrated in FIG. 7. The logical connections depicted in FIG. 7 include a local area network (LAN) 571 and a wide area network (WAN) 573, but may also include other networks. Such networking environments are commonplace in hospitals, offices, enterprise-wide computer networks, intranets and the Internet.

When used in a LAN networking environment, the computer 510 is connected to the LAN 571 through a network interface or adapter 570. When used in a WAN networking environment, the computer 510 typically includes a modem 572 or other means for establishing communications over the WAN 573, such as the Internet. The modem 572, which may be internal or external, may be connected to the system bus 521 via the input interface 560, or via another appropriate mechanism. In a networked environment, program modules depicted relative to the computer 510, or portions thereof, may be stored in the remote memory storage device 581. By way of example, and not limitation, FIG. 7 illustrates remote application programs 585 as residing on memory device 581.

The communications connections 570, 572 allow the device to communicate with other devices. The communications connections 570, 572 are an example of communication media, as discussed above.

The methods of any of the embodiments described above (e.g., methods 300, 320, 340, 400, and/or 420) may be implemented wholly or in part using one or more computer systems such as the computer system 500 illustrated in FIG. 7. Referring generally to the embodiments of FIG. 1, for example, the computer 510 may be used as some or all of computer system 16, with the units 80 and 82 being instructions that are a part of application programs 535 stored in RAM 532 and/or application programs 545 stored in hard disk drive 541. As another example, data from on-board system 14 may be received via a modem similar to the modem 572, which may in turn be coupled to a network similar to network 20 of FIG. 1.

TECHNICAL ADVANTAGES

The aspects described herein may be implemented as part of one or more computer components, such a server device, for example. Furthermore, the aspects described herein may be implemented within a computer network architecture implementing vehicle telematics technology, and may leverage that architecture and technology to obtain new and beneficial results not previously achieved. Thus, the aspects described herein address and solve issues of a technical nature that are necessarily rooted in computer technology.

For instance, aspects described herein may include analyzing various sources of vehicle data to identify certain driving behaviors that are not captured or recognized by conventional systems, such as operator responsiveness to ADAS alerts. Without the improvements provided by capturing such driving behaviors, the assessment of driving behavior as it pertains to ADAS-installed vehicles would not be accurate, or may require much larger samples of telematics data to be collected and processed. Naturally, this would result in additional memory usage, processing resources, and/or time. Thus, aspects described herein address computer-related issues that are related to efficiency, processing, and storage metrics, such as consuming less power and/or memory, for example.

ADDITIONAL CONSIDERATIONS

With the foregoing, an insurance customer may opt-in to a rewards, insurance discount, or other type of program. After the insurance customer provides their affirmative consent, an insurance provider remote server may collect data from the customer's mobile device, smart vehicle, autonomous or semi-autonomous vehicle, smart home controller, or other smart devices—such as with the customer's permission or affirmative consent. The data collected may be related to smart or autonomous vehicle functionality, smart home functionality (or home occupant preferences or preference profiles), and/or insured assets before (and/or after) an insurance-related event, including those events discussed elsewhere herein. In return, those insured may receive discounts or insurance cost savings related to auto, home, renters, personal articles, mobile, and other types of insurance from the insurance provider.

The patent claims at the end of this patent application are not intended to be construed under 35 U.S.C. § 112(f) unless traditional means-plus-function language is expressly recited, such as “means for” or “step for” language being explicitly recited in the claim(s). The systems and methods described herein are directed to an improvement to computer functionality, and improve the functioning of conventional computers.

The following considerations also apply to the foregoing discussion. Throughout this specification, plural instances may implement operations or structures described as a single instance. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations may be performed concurrently, and nothing requires that the operations be performed in the order illustrated. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter herein.

Unless specifically stated otherwise, discussions herein using words such as “processing,” “computing,” “calculating,” “determining,” “presenting,” “displaying,” or the like may refer to actions or processes of a machine (e.g., a computer) that manipulates or transforms data represented as physical (e.g., electronic, magnetic, or optical) quantities within one or more memories (e.g., volatile memory, non-volatile memory, or a combination thereof), registers, or other machine components that receive, store, transmit, or display information.

As used herein any reference to “one embodiment” or “an embodiment” means that a particular element, feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment.

As used herein, the terms “comprises,” “comprising,” “includes,” “including,” “has,” “having” or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Further, unless expressly stated to the contrary, “or” refers to an inclusive or and not to an exclusive or. For example, a condition A or B is satisfied by any one of the following: A is true (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present).

In addition, use of “a” or “an” is employed to describe elements and components of the embodiments herein. This is done merely for convenience and to give a general sense of the invention. This description should be read to include one or at least one and the singular also includes the plural unless it is obvious that it is meant otherwise.

Upon reading this disclosure, those of skill in the art will appreciate still additional alternative structural and functional designs for generating, modifying, and/or using driver profiles through the principles disclosed herein. Thus, while particular embodiments and applications have been illustrated and described, it is to be understood that the disclosed embodiments are not limited to the precise construction and components disclosed herein. Various modifications, changes and variations, which will be apparent to those skilled in the art, may be made in the arrangement, operation and details of the method and apparatus disclosed herein without departing from the spirit and scope defined in the appended claims. 

1. A computer-implemented method, carried out by a processor, for detecting and acting upon operator responsiveness to vehicle alerts, the method comprising: determining, by the processor, that an Advanced Driver Assistance System (ADAS) of a vehicle generated an alert for a driving activity and an alert timestamp; logging, by the processor, a first set of measurements data indicative of the driving activity and including vehicle operational data that caused the activation of the alert of the ADAS and a timestamp of when the alert of the ADAS has been activated in a database saved in a memory; receiving, by the processor from a plurality of sensors, a second set of measurements data in real time in response to the alert for the driving activity, the plurality of sensors comprising a lidar device, the second set of measurements data comprising digital lidar information collected from the lidar device; comparing, by the processor, a portion of the first set of measurements data with a portion of the second set of measurements data; determining, by the processor, a driving metric for an operator of the vehicle based upon the comparing; and setting, by the processor, at least a portion of an operator profile associated with the operator with the driving metric, wherein determining the driving metric comprises: determining whether the alert of the ADAS is classified as a false positive alert or a valid alert based upon the comparing; and adjusting the driving metric based upon whether the alert is classified as the false positive alert or the valid alert and whether the alert of the ADAS remains activated after a predetermined amount of time after the alert timestamp.
 2. The computer-implemented method of claim 1, further comprising: causing, by the processor, transmission of at least the portion of the operator profile to an entity that: adjusts a price to risk model associated with the operator based upon at least the portion of the operator profile, adjusts a credit rating associated with the operator based upon at least the portion of the operator profile, adjusts an insurance rating associated with the operator based upon at least the portion of the operator profile, reviews at least the portion of the operator profile in connection with a job sought by the operator, or offers a permanent or temporary credit, in connection with a good or service offered by the entity, based upon at least the portion of the operator profile.
 3. The computer-implemented method of claim 1, further comprising: adjusting, by the processor, at least one of a price to risk model, a credit rating, an insurance rating, a review, a permanent credit, or a temporary credit associated with the operator based upon at least the portion of the operator profile.
 4. The computer-implemented method of claim 1, wherein the first set of measurements data comprises data indicative of at least one of: whether ADAS has been engaged in the vehicle, whether a particular feature of the ADAS has been operational, or whether the alert of the ADAS has been activated, and wherein the second set of measurements data comprises data associated with a pre-determinable amount of time that has elapsed since the timestamp of when the alert of the ADAS has been activated, the data indicative of at least one of: whether the alert of the ADAS is still activated, or whether the operational data that caused the alert of the ADAS to activate has remained consistent.
 5. The computer-implemented method of claim 4, wherein adjusting the driving metric further comprises lowering the driving metric if: (i) the alert is classified as the false positive alert based upon the first set of measurements data, and the portion of the second set of measurements data is not consistent with the portion of the first set of measurements data; (ii) the alert is classified as the false positive alert based upon the first set of measurements data, and the alert no longer remains indicated to the operator of the vehicle based upon the second set of measurements data; (iii) the alert is classified as the valid alert based upon the first set of measurements data, and the portion of the second set of measurements data is consistent with the portion of the first set of measurements data; or (iv) the alert is classified as the valid alert based upon the first set of measurements data, and the alert remains indicated to the operator of the vehicle based upon the second set of measurements data.
 6. The computer-implemented method of claim 4, wherein adjusting the driving metric further comprises increasing the driving metric if: (i) the alert is classified as the false positive alert based upon the first set of measurements data, and the portion of the second set of measurements data is consistent with the portion of the first set of measurements data; (ii) the alert is classified as the false positive alert based upon the first set of measurements data, and the alert remains indicated to the operator of the vehicle based upon the second set of measurements data; (iii) the alert is classified as the valid alert based upon the first set of measurements data, and the portion of the second set of measurements data is not consistent with the portion of the first set of measurements data; or (iv) the alert is classified as the valid alert based upon the first set of measurements data, and the alert no longer remains indicated to the operator of the vehicle based upon the second set of measurements data.
 7. The computer-implemented method of claim 1, wherein logging the first set of measurements data includes: receiving operational data generated by a subsystem of the vehicle, wherein the operational data comprises data relating to operation metrics of the vehicle, indicative of at least one of: speed, acceleration, braking, weaving, steering, ADAS, drive mode, headlight, turn signal, or windshield wiper.
 8. A computer system for detecting and acting upon operator responsiveness to vehicle alerts, the computer system comprising: one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the computer system to: determine that an Advanced Driver Assistance System (ADAS) of a vehicle generated an alert for a driving activity and an alert timestamp; log a first set of measurements data indicative of the driving activity and including vehicle operational data that caused the activation of the alert of the ADAS and a timestamp of when the alert of the ADAS has been activated in a database saved in a memory; receive a second set of measurements data in real time in response to the alert for the driving activity from a plurality of sensors, the plurality of sensors comprising a lidar device, the second set of measurements data comprising digital lidar information collected from the lidar device; compare a portion of the first set of measurements data with a portion of the second set of measurements data; determine a driving metric for an operator of the vehicle based upon the comparing; and set at least a portion of an operator profile associated with the operator with the driving metric, wherein to determine the driving metric comprises: determining whether the alert of the ADAS is classified as a false positive alert or a valid alert based upon the comparing; and adjusting the driving metric based upon whether the alert is classified as the false positive alert or the valid alert and whether the alert of the ADAS remains activated after a predetermined amount of time after the alert timestamp.
 9. The computer system of claim 8, wherein the instructions further cause the computer system to transmit at least the portion of the operator profile to an entity that: adjusts a price to risk model associated with the operator based upon at least the portion of the operator profile, adjusts a credit rating associated with the operator based upon at least the portion of the operator profile, adjusts an insurance rating associated with the operator based upon at least the portion of the operator profile, reviews at least the portion of the operator profile in connection with a job sought by the operator, or offers a permanent or temporary credit, in connection with a good or service offered by the entity, based upon at least the portion of the operator profile.
 10. The computer system of claim 8, wherein the instructions further cause the computer system to adjust at least one of a price to risk model, a credit rating, an insurance rating, a review, a permanent credit, or a temporary credit associated with the operator based upon at least the portion of the operator profile.
 11. The computer system of claim 8, wherein the first set of measurements data comprises data indicative of at least one of: whether ADAS has been engaged in the vehicle, whether a particular feature of the ADAS has been operational, or whether the alert of the ADAS has been activated, wherein the second set of measurements data comprises data associated with a pre-determinable amount of time that has elapsed since the timestamp of when the alert of the ADAS has been activated, the data indicative of at least one of: whether the alert of the ADAS is still activated, or whether the operational data that caused the alert of the ADAS to activate has remained consistent.
 12. The computer system of claim 11, wherein to adjust the driving metric comprises to adjust the driving metric by lowering the driving metric if: (i) the alert is classified as the false positive alert based upon the first set of measurements data, and the portion of the second set of measurements data is not consistent with the portion of the first set of measurements data; (ii) the alert is classified as the false positive alert based upon the first set of measurements data, and the alert no longer remains indicated to the operator of the vehicle based upon the second set of measurements data; (iii) the alert is classified as the valid alert based upon the first set of measurements data, and the portion of the second set of measurements data is consistent with the portion of the first set of measurements data; or (iv) the alert is classified as the valid alert based upon the first set of measurements data, and the alert remains indicated to the operator of the vehicle based upon the second set of measurements data.
 13. The computer system of claim 11, wherein to adjust the driving metric comprises to adjust the driving metric by increasing the driving metric if: (i) the alert is classified as the false positive alert based upon the first set of measurements data, and the portion of the second set of measurements data is consistent with the portion of the first set of measurements data; (ii) the alert is classified as the false positive alert based upon the first set of measurements data, and the alert remains indicated to the operator of the vehicle based upon the second set of measurements data; (iii) the alert is classified as the valid alert based upon the first set of measurements data, and the portion of the second set of measurements data is not consistent with the portion of the first set of measurements data; or (iv) the alert is classified as the valid alert based upon the first set of measurements data, and the alert no longer remains indicated to the operator of the vehicle based upon the second set of measurements data.
 14. The computer system of claim 8, wherein the instructions cause the computer system to log the first set of measurements data at least by receiving operational data generated by a subsystem of the vehicle, wherein the operational data comprises data relating to operation metrics of the vehicle, indicative of at least one of: speed, acceleration, braking, weaving, steering, ADAS, drive mode, headlight, turn signal, or windshield wiper.
 15. A non-transitory, computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to: determine that an Advanced Driver Assistance System (ADAS) of a vehicle generated an alert for a driving activity and an alert timestamp; log a first set of measurements data indicative of the driving activity and including vehicle operational data that caused the activation of the alert of the ADAS and a timestamp of when the alert of the ADAS has been activated in a database saved in a memory; receive a second set of measurements data in real time in response to the alert for the driving activity from a plurality of sensors, the plurality of sensors comprising a lidar device, the second set of measurements data comprising digital lidar information collected from the lidar device; compare a portion of the first set of measurements data with a portion of the second set of measurements data; determine a driving metric for an operator of the vehicle based upon the comparing; and set at least a portion of an operator profile associated with the operator with the driving metric, wherein to determine the driving metric comprises: determining whether the alert of the ADAS is classified as a false positive alert or a valid alert based upon the comparing; and adjusting the driving metric based upon whether the alert is classified as the false positive alert or the valid alert and whether the alert of the ADAS remains activated after a predetermined amount of time after the alert timestamp.
 16. The non-transitory, computer-readable medium of claim 15, wherein the instructions further cause the one or more processors to transmit at least the portion of the operator profile to an entity that: adjusts a price to risk model associated with the operator based upon at least the portion of the operator profile, adjusts a credit rating associated with the operator based upon at least the portion of the operator profile, adjusts an insurance rating associated with the operator based upon at least the portion of the operator profile, reviews at least the portion of the operator profile in connection with a job sought by the operator, or offers a permanent or temporary credit, in connection with a good or service offered by the entity, based upon at least the portion of the operator profile.
 17. The non-transitory, computer-readable medium of claim 15, wherein the instructions further cause the one or more processors to adjust at least one of a price to risk model, a credit rating, an insurance rating, a review, a permanent credit, or a temporary credit associated with the operator based upon at least the portion of the operator profile.
 18. The non-transitory, computer-readable medium of claim 15, wherein the first set of measurements data comprises data indicative of at least one of: whether ADAS has been engaged in the vehicle, whether a particular feature of the ADAS has been operational, or whether the alert of the ADAS has been activated, and wherein the second set of measurements data comprises data associated with a pre-determinable amount of time that has elapsed since the timestamp of when the alert of the ADAS has been activated, the data indicative of at least one of: whether the alert of the ADAS is still activated, or whether the operational data that caused the alert of the ADAS to activate has remained consistent.
 19. The non-transitory, computer-readable medium of claim 18, wherein to adjust the driving metric comprises to adjust driving metric by lowering the driving metric if: (i) the alert is classified as the false positive alert based upon the first set of measurements data, and the portion of the second set of measurements data is not consistent with the portion of the first set of measurements data; (ii) the alert is classified as the false positive alert based upon the first set of measurements data, and the alert no longer remains indicated to the operator of the vehicle based upon the second set of measurements data; (iii) the alert is classified as the valid alert based upon the first set of measurements data, and the portion of the second set of measurements data is consistent with the portion of the first set of measurements data; or (iv) the alert is classified as the valid alert based upon the first set of measurements data, and the alert remains indicated to the operator of the vehicle based upon the second set of measurements data.
 20. The non-transitory, computer-readable medium of claim 18, wherein to adjust the driving metric comprises to adjust driving metric by increasing the driving metric if: (i) the alert is classified as the false positive alert based upon the first set of measurements data, and the portion of the second set of measurements data is consistent with the portion of the first set of measurements data; (ii) the alert is classified as the false positive alert based upon the first set of measurements data, and the alert remains indicated to the operator of the vehicle based upon the second set of measurements data; (iii) the alert is classified as the valid alert based upon the first set of measurements data, and the portion of the second set of measurements data is not consistent with the portion of the first set of measurements data; or (iv) the alert is classified as the valid alert based upon the first set of measurements data, and the alert no longer remains indicated to the operator of the vehicle based upon the second set of measurements data. 